24/03/2024 -30/03/2024

26/03/2024 05:38

It seems putting in the new SFPs fixed our error with the master frontend, I can now get it to start up successfully:

[root@dhcp-10-163-105-238 MasterGM2]# ./frontend -e DAQ
Frontend name          :     MasterGM2
Event buffer size      :     1000000
User max event size    :     10000
# of events per buffer :     100

Connect to experiment DAQ...
OK
Init hardware...
print(155): /Equipment/MasterGM2/Settings/Globals Trigger source                   GPS
print(156): /Equipment/MasterGM2/Settings/Globals Socket trigger IP address        127.0.0.1
print(157): /Equipment/MasterGM2/Settings/Globals Socket trigger port              55000
print(158): /Equipment/MasterGM2/Settings/Globals Rate                             12.000000
print(159): /Equipment/MasterGM2/Settings/Globals Fills per bunch                  8
print(160): /Equipment/MasterGM2/Settings/Globals Readout name                     AMC13
print(161): /Equipment/MasterGM2/Settings/Globals Front End Offset                 1
print(162): /Equipment/MasterGM2/Settings/Globals Simulator name                   CaloSimulatorAMC13
print(163): /Equipment/MasterGM2/Settings/Globals Encoder Front End                AMC13001
print(164): /Equipment/MasterGM2/Settings/Globals Send to event builder            1
print(165): /Equipment/MasterGM2/Settings/Globals Verbose                          0
print(166): /Equipment/MasterGM2/Settings/Globals CCC                              1
print(167): /Equipment/MasterGM2/Settings/Globals Post MTCA Abort State to ECL     1
print(168): /Equipment/MasterGM2/Settings/Globals Preserve TTC Trigger Config      0
print(169): /Equipment/MasterGM2/Settings/Globals Preserve Analog Trigger Config   0
print(170): /Equipment/MasterGM2/Settings/Globals Enable Trigger DB                1
print(171): /Equipment/MasterGM2/Settings/Globals Trigger DB Analog A6 ID          1
print(172): /Equipment/MasterGM2/Settings/Globals Trigger DB Analog T9 ID          1
print(173): /Equipment/MasterGM2/Settings/Globals Trigger DB TTC ID                1
print(174): /Equipment/MasterGM2/Settings/Globals Trigger DB TTC Analog Pulse ID   1
print(175): /Equipment/MasterGM2/Settings/Globals Trigger DB Fanout Delays ID      1
print(176): /Equipment/MasterGM2/Settings/Globals Database Connection              pgsql://g2db-priv:5433/gm2_online_prod gm2_writer gm2_4_writer
print(177): /Equipment/MasterGM2/Settings/Globals Internal Trig Alarm Thresh (s)   30
print(178): /Equipment/MasterGM2/Settings/Globals T9A6Gap (ns)                     8654175
get_slave_info : 1 enabled slave frontends:
    FE [AMC13001] : slave index = 0 : sync = 0
Key: /Equipment/AMC13001/Common/Enabled
Looking for encoder in crate 1
Found encoder FC7 in slot 11
/AMC13XX/Settings/: ODB Verifcation
Looking for encoder in crate 1
MCH IPMI Communication Check
AMC13 Initialization Started
AMC13_init(78): Read AMC13 T1 IP Address: 192.168.1.189
AMC13_init(86): Read AMC13 T2 IP Address: 192.168.1.188
ipbusudp-2.0://192.168.1.189:50001 , file://$GM2DAQ_DIR/address_tables/AMC13XG_T1.xml
AMC13_init(120): AMC13 T1 FPGA Firmware Version Check
AMC13_init(135): AMC13 T2 FPGA Firmware Version Check
AMC13_init(159): AMC13 Backplane Enables
FC7 Initialization Started
FC7_init(174): FC7 Board Presence Check
FC7_init(190): Slot 11: Read FC7 IP Address: 192.168.1.11
FC7_init(241): FC7 Ethernet Communication Check: 1/1
FC7_init(288): Slot 11: FC7 Firmware Hard Reset
FC7_init(298): Waiting 5 s ...
FC7_init(410): Slot 11: Write: Enabled Top SFP Ports
top: 1, enable: 1
FC7_init(420): Waiting 15 s ...
FC7_init(454): Slot 11: Read: Enabled Top SFP Ports: 1
FC7_init(473): Slot 11: Write: Top TTS Receiver Reset
MicroTCA initialization duration: 21.866452 s
frontend_init: Trigger source is GPS
trigger source 3 GPS 3
Name of Device Found: TCR180PEX
Buffers Used:   0, Buffers Capacity: 584
Frontend Initialization Complete
OK
[root@dhcp-10-163-105-238 MasterGM2]# ./frontend -e DAQ
Frontend name          :     MasterGM2
Event buffer size      :     1000000
User max event size    :     10000
# of events per buffer :     100

Connect to experiment DAQ...
OK
Init hardware...
print(155): /Equipment/MasterGM2/Settings/Globals Trigger source                   GPS
print(156): /Equipment/MasterGM2/Settings/Globals Socket trigger IP address        127.0.0.1
print(157): /Equipment/MasterGM2/Settings/Globals Socket trigger port              55000
print(158): /Equipment/MasterGM2/Settings/Globals Rate                             12.000000
print(159): /Equipment/MasterGM2/Settings/Globals Fills per bunch                  8
print(160): /Equipment/MasterGM2/Settings/Globals Readout name                     AMC13
print(161): /Equipment/MasterGM2/Settings/Globals Front End Offset                 1
print(162): /Equipment/MasterGM2/Settings/Globals Simulator name                   CaloSimulatorAMC13
print(163): /Equipment/MasterGM2/Settings/Globals Encoder Front End                AMC13001
print(164): /Equipment/MasterGM2/Settings/Globals Send to event builder            1
print(165): /Equipment/MasterGM2/Settings/Globals Verbose                          0
print(166): /Equipment/MasterGM2/Settings/Globals CCC                              1
print(167): /Equipment/MasterGM2/Settings/Globals Post MTCA Abort State to ECL     1
print(168): /Equipment/MasterGM2/Settings/Globals Preserve TTC Trigger Config      0
print(169): /Equipment/MasterGM2/Settings/Globals Preserve Analog Trigger Config   0
print(170): /Equipment/MasterGM2/Settings/Globals Enable Trigger DB                1
print(171): /Equipment/MasterGM2/Settings/Globals Trigger DB Analog A6 ID          1
print(172): /Equipment/MasterGM2/Settings/Globals Trigger DB Analog T9 ID          1
print(173): /Equipment/MasterGM2/Settings/Globals Trigger DB TTC ID                1
print(174): /Equipment/MasterGM2/Settings/Globals Trigger DB TTC Analog Pulse ID   1
print(175): /Equipment/MasterGM2/Settings/Globals Trigger DB Fanout Delays ID      1
print(176): /Equipment/MasterGM2/Settings/Globals Database Connection              pgsql://g2db-priv:5433/gm2_online_prod gm2_writer gm2_4_writer
print(177): /Equipment/MasterGM2/Settings/Globals Internal Trig Alarm Thresh (s)   30
print(178): /Equipment/MasterGM2/Settings/Globals T9A6Gap (ns)                     8654175
get_slave_info : 1 enabled slave frontends:
    FE [AMC13001] : slave index = 0 : sync = 0
Key: /Equipment/AMC13001/Common/Enabled
Looking for encoder in crate 1
Found encoder FC7 in slot 11
/AMC13XX/Settings/: ODB Verifcation
Looking for encoder in crate 1
MCH IPMI Communication Check
AMC13 Initialization Started
AMC13_init(78): Read AMC13 T1 IP Address: 192.168.1.189
AMC13_init(86): Read AMC13 T2 IP Address: 192.168.1.188
ipbusudp-2.0://192.168.1.189:50001 , file://$GM2DAQ_DIR/address_tables/AMC13XG_T1.xml
AMC13_init(120): AMC13 T1 FPGA Firmware Version Check
AMC13_init(135): AMC13 T2 FPGA Firmware Version Check
AMC13_init(159): AMC13 Backplane Enables
FC7 Initialization Started
FC7_init(174): FC7 Board Presence Check
FC7_init(190): Slot 11: Read FC7 IP Address: 192.168.1.11
FC7_init(241): FC7 Ethernet Communication Check: 1/1
FC7_init(288): Slot 11: FC7 Firmware Hard Reset
FC7_init(298): Waiting 5 s ...
FC7_init(410): Slot 11: Write: Enabled Top SFP Ports
top: 1, enable: 1
FC7_init(420): Waiting 15 s ...
FC7_init(454): Slot 11: Read: Enabled Top SFP Ports: 1
FC7_init(473): Slot 11: Write: Top TTS Receiver Reset
MicroTCA initialization duration: 21.866452 s
frontend_init: Trigger source is GPS
trigger source 3 GPS 3
Name of Device Found: TCR180PEX
Buffers Used:   0, Buffers Capacity: 584
Frontend Initialization Complete
OK

26/03/2024 05:43

I changed the IP of the WFD5 so it "matches" it's slot number now instead:

[root@dhcp-10-163-105-238 software]# python store_ip.py 1 5 192.168.1.5
ipmitool -I lan -H 192.168.1.41 -U '' -P '' -m 0x20 -B 0x0 -T 0x82 -b 7 -t 0x7a raw 0x32 0x51 0x05 0x01 0xa8 0xc0
[root@dhcp-10-163-105-238 software]# python store_ip.py 1 5 192.168.1.5
ipmitool -I lan -H 192.168.1.41 -U '' -P '' -m 0x20 -B 0x0 -T 0x82 -b 7 -t 0x7a raw 0x32 0x51 0x05 0x01 0xa8 0xc0

physically pull black hotswap switch on the WFD5
wait for "mgmt" light to be the only green line
physicall push black hotswap switch ojn the WFD5 back in

[root@dhcp-10-163-105-238 software]# ping 192.168.1.5
PING 192.168.1.5 (192.168.1.5) 56(84) bytes of data.
64 bytes from 192.168.1.5: icmp_seq=1 ttl=64 time=0.135 ms
64 bytes from 192.168.1.5: icmp_seq=2 ttl=64 time=0.060 ms
^C
--- 192.168.1.5 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 999ms
rtt min/avg/max/mdev = 0.060/0.097/0.135/0.038 ms
[root@dhcp-10-163-105-238 software]#
[root@dhcp-10-163-105-238 software]# ping 192.168.1.5
PING 192.168.1.5 (192.168.1.5) 56(84) bytes of data.
64 bytes from 192.168.1.5: icmp_seq=1 ttl=64 time=0.135 ms
64 bytes from 192.168.1.5: icmp_seq=2 ttl=64 time=0.060 ms
^C
--- 192.168.1.5 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 999ms
rtt min/avg/max/mdev = 0.060/0.097/0.135/0.038 ms
[root@dhcp-10-163-105-238 software]#

26/03/2024 06:06

Just a "note to self" when 'fe01' is plugged into LAN, it has IP:

10.163.102.46
10.163.102.46

So you can connect with

ssh root@10.163.102.46
ssh root@10.163.102.46

26/03/2024 06:08

Just as a sanity check, I made just the page size of 'fe01' is 4KB as it is on most machine:

[root@fe01 ~]# getconf PAGE_SIZE
4096
[root@fe01 ~]# getconf PAGE_SIZE
4096

So this 512 4-byte "periodic" structure of the data I read is still a mystery to me.

26/03/2024 06:18

This is as far as I get with the CaloReadoutAMC13 frontend:

[root@dhcp-10-163-105-238 CaloReadoutAMC13]# ./frontend -i 1 -e DAQ
Frontend name          :     AMC13001
Event buffer size      :     4194304
User max event size    :     2097152
# of events per buffer :     2

Connect to experiment DAQ...
OK
Init hardware...
Frontend Index = 1, IP Offset = 1
totalSlots = 2
Enabled Mask (Binary): 010000010000
frontend_init(1754): AMC13 ODB Verifcation
frontend_init(1762): WFD5 ODB Verifcation
frontend_init(1768): FC7 ODB Verifcation
frontend_init(1778): MCH IPMI Communication Check
frontend_init(1803): Read AMC13 T1 IP Address: 192.168.1.189
frontend_init(1811): Read AMC13 T2 IP Address: 192.168.1.188
frontend_init(1840): AMC13 T1 FPGA Firmware Version Check
frontend_init(1851): AMC13 T2 FPGA Firmware Version Check
frontend_init(1862): AMC13 TTC Signal Presence Check
[AMC13001,ERROR] [frontend.cpp:1866:frontend_init,ERROR] AMC13: TTC Signal Absent
[root@dhcp-10-163-105-238 CaloReadoutAMC13]# ./frontend -i 1 -e DAQ
Frontend name          :     AMC13001
Event buffer size      :     4194304
User max event size    :     2097152
# of events per buffer :     2

Connect to experiment DAQ...
OK
Init hardware...
Frontend Index = 1, IP Offset = 1
totalSlots = 2
Enabled Mask (Binary): 010000010000
frontend_init(1754): AMC13 ODB Verifcation
frontend_init(1762): WFD5 ODB Verifcation
frontend_init(1768): FC7 ODB Verifcation
frontend_init(1778): MCH IPMI Communication Check
frontend_init(1803): Read AMC13 T1 IP Address: 192.168.1.189
frontend_init(1811): Read AMC13 T2 IP Address: 192.168.1.188
frontend_init(1840): AMC13 T1 FPGA Firmware Version Check
frontend_init(1851): AMC13 T2 FPGA Firmware Version Check
frontend_init(1862): AMC13 TTC Signal Presence Check
[AMC13001,ERROR] [frontend.cpp:1866:frontend_init,ERROR] AMC13: TTC Signal Absent

We have the TTC signal input, my guess is we need to update the firmware.

26/03/2024 16:46

I changed the priority for the 10GbE port so it stops stealing traffic from our 1GbE port:

#
# Connect to AMC
#
TYPE=Ethernet
BOOTPROTO=static
IPADDR=192.168.1.20
NETMASK=255.255.255.0
IPV4_FAILURE_FATAL=no
IPV6INIT=no
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_PEERDNS=yes
IPV6_PEERROUTES=yes
IPV6_FAILURE_FATAL=no
NAME=enp1s0f1
DEVICE=enp1s0f1
ONBOOT=yes
AUTOCONNECT_PRIORITY=-999
MTU=9600
#
# Connect to AMC
#
TYPE=Ethernet
BOOTPROTO=static
IPADDR=192.168.1.20
NETMASK=255.255.255.0
IPV4_FAILURE_FATAL=no
IPV6INIT=no
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_PEERDNS=yes
IPV6_PEERROUTES=yes
IPV6_FAILURE_FATAL=no
NAME=enp1s0f1
DEVICE=enp1s0f1
ONBOOT=yes
AUTOCONNECT_PRIORITY=-999
MTU=9600

And similar for enp1s0f0

#
# Connect to AMC
#
TYPE=Ethernet
BOOTPROTO=static
IPADDR=192.168.1.21
NETMASK=255.255.255.0
IPV4_FAILURE_FATAL=no
IPV6INIT=no
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_PEERDNS=yes
IPV6_PEERROUTES=yes
IPV6_FAILURE_FATAL=no
NAME=enp1s0f0
DEVICE=enp1s0f0
ONBOOT=yes
AUTOCONNECT_PRIORITY=-999
MTU=9600
#
# Connect to AMC
#
TYPE=Ethernet
BOOTPROTO=static
IPADDR=192.168.1.21
NETMASK=255.255.255.0
IPV4_FAILURE_FATAL=no
IPV6INIT=no
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_PEERDNS=yes
IPV6_PEERROUTES=yes
IPV6_FAILURE_FATAL=no
NAME=enp1s0f0
DEVICE=enp1s0f0
ONBOOT=yes
AUTOCONNECT_PRIORITY=-999
MTU=9600

I tried to ping 192.168.1.25 as mentioned in Tim's notes:

4) CONFIGURE THE AMC13

bin/AMC13Tool -u
do amc13_scripts/ky_init.amc

# check the 10 GbE link is up
ping 192.168.1.32
PING 192.168.1.32 (192.168.1.32) 56(84) bytes of data.
64 bytes from 192.168.1.32: icmp_seq=1 ttl=64 time=0.038 ms
64 bytes from 192.168.1.32: icmp_seq=2 ttl=64 time=0.021 ms

4b) if 4) doesnt work
if ping doesn't work try 
# reset of AMC13 
bin/AMC13Tool -u
ws 0x0 0x10 
then repeat 3) and 4) - this worked for me on 4/22/15

5) RUN SIMPLE CLIENT
cd /home/daq/DAQ/amc13/old_version/examples/SimpleClient

./client 192.168.1.32
Starting run
dt = 0 s 0 us
4) CONFIGURE THE AMC13

bin/AMC13Tool -u
do amc13_scripts/ky_init.amc

# check the 10 GbE link is up
ping 192.168.1.32
PING 192.168.1.32 (192.168.1.32) 56(84) bytes of data.
64 bytes from 192.168.1.32: icmp_seq=1 ttl=64 time=0.038 ms
64 bytes from 192.168.1.32: icmp_seq=2 ttl=64 time=0.021 ms

4b) if 4) doesnt work
if ping doesn't work try 
# reset of AMC13 
bin/AMC13Tool -u
ws 0x0 0x10 
then repeat 3) and 4) - this worked for me on 4/22/15

5) RUN SIMPLE CLIENT
cd /home/daq/DAQ/amc13/old_version/examples/SimpleClient

./client 192.168.1.32
Starting run
dt = 0 s 0 us

but was unable to.

I disabled every port on the network with ifdown enp5s0 and enp1s0f0 to make sure they aren't stealing traffic. Then I tried pinging 192.168.1.32 but was unable to. I feel like this port needs to be "configured" somehow.

26/03/2024 16:55

The priority thing is a bit finnicky. If I ifdown enp5s0 then ifup enp5s0 I find that somehow enp1s0f1 takes priority. So I have to ifdown enp1s0f1 then ifup enp1s0f1 to make the priority order "correct" again. You can check priority order with this command:

[root@dhcp-10-163-105-238 network-scripts]# ip route show
default via 10.163.105.1 dev eno1 proto dhcp metric 101
10.163.105.0/24 dev eno1 proto kernel scope link src 10.163.105.238 metric 101
192.168.1.0/24 dev enp5s0 proto kernel scope link src 192.168.1.100 metric 103
192.168.1.0/24 dev enp1s0f1 proto kernel scope link src 192.168.1.20 metric 104
192.168.122.0/24 dev virbr0 proto kernel scope link src 192.168.122.1
[root@dhcp-10-163-105-238 network-scripts]# ip route show
default via 10.163.105.1 dev eno1 proto dhcp metric 101
10.163.105.0/24 dev eno1 proto kernel scope link src 10.163.105.238 metric 101
192.168.1.0/24 dev enp5s0 proto kernel scope link src 192.168.1.100 metric 103
192.168.1.0/24 dev enp1s0f1 proto kernel scope link src 192.168.1.20 metric 104
192.168.122.0/24 dev virbr0 proto kernel scope link src 192.168.122.1

Higher metric number corresponds to worse priority.

26/03/2024 17:16

It looks like there was port configuration in ky_init.amc

[root@dhcp-10-163-105-238 amc13_scripts]# cat ky_init.amc
#
# This AMC13Tool script initializes the AMC13 from all AMC inputs and
# prepares the module to send local L1As and build fake events
#
# Reset the AMC13 (not usually necessary, but never a bad idea)
rg
# Enable the AMC13 from all twelve of its AMC inputs,
# enable run mode, and prepare to take fake data
en 0-11 f
# Display AMC13 status
st
# setup
# run control, only uses bits 0-14
wv 0x1 0x0
# enables amc slots and SFP outputs
wv 0x3 0x1fff
# run control, 0x187 means
# bit 0 '1' run mode, bit 1 '1' enables DAQLSC, bit 2 "1" use LIA
# bit 7 "1" generate fake event on receiving L1A, bit 8 "1" TTs=TTC
wv 0x1 0x187
# controls the rate in free running mode using lower 16 bits
# 0x8000 is period of L1As so decreasing 0x8000 increases rate
wv 0x1c 0x80008000
# 0x550 makes 32kB events, 13 May 2014 changed to 17 bit payload
#wv 0x18 0x550
# 0x550 makes 25MB events, 13 May 2014 changed to 17 bit max payload
wv 0x18 0x3fff0
# bit 5 '1' reset ddr3 memory controller
wv 0x0 0x10
# wv 0x0 0x1
# (exit this script file)
#q
[root@dhcp-10-163-105-238 amc13_scripts]# cat ky_init.amc
#
# This AMC13Tool script initializes the AMC13 from all AMC inputs and
# prepares the module to send local L1As and build fake events
#
# Reset the AMC13 (not usually necessary, but never a bad idea)
rg
# Enable the AMC13 from all twelve of its AMC inputs,
# enable run mode, and prepare to take fake data
en 0-11 f
# Display AMC13 status
st
# setup
# run control, only uses bits 0-14
wv 0x1 0x0
# enables amc slots and SFP outputs
wv 0x3 0x1fff
# run control, 0x187 means
# bit 0 '1' run mode, bit 1 '1' enables DAQLSC, bit 2 "1" use LIA
# bit 7 "1" generate fake event on receiving L1A, bit 8 "1" TTs=TTC
wv 0x1 0x187
# controls the rate in free running mode using lower 16 bits
# 0x8000 is period of L1As so decreasing 0x8000 increases rate
wv 0x1c 0x80008000
# 0x550 makes 32kB events, 13 May 2014 changed to 17 bit payload
#wv 0x18 0x550
# 0x550 makes 25MB events, 13 May 2014 changed to 17 bit max payload
wv 0x18 0x3fff0
# bit 5 '1' reset ddr3 memory controller
wv 0x0 0x10
# wv 0x0 0x1
# (exit this script file)
#q

particularly this line:

# enables amc slots and SFP outputs
wv 0x3 0x1fff
# enables amc slots and SFP outputs
wv 0x3 0x1fff

27/03/2024 20:30

From the mysterious Mr. Wu's notes we have:

Note from Mr Wu

MSS applies only to TCP, ping is ICMP so is irrelevant. amc13 does not accept any TCP data, so its receiving buffer
has a limited size, if the buffer is full, packet will be dropped.
We noticed that NIC has a tendency of exaggerating its receiving window, this leads to dropping the packets and
repeated resending on the amc13 side. amc13 can set an upper limit on its sending window by writing to reg 0x1c1f
e.g. 0x20000 which limits the number of packets in the tube to 32. Depending on the RTT, this may limit the bandwidth
of the TCP, so try to use a smaller number if it can meet the 2.7Gb/s requirement

list of TCP registers in AMC13.

0x1c00-0x1fff TCPIP interface general registers
0x1c0d event cmsCRC error counter for event builder 0
0x1c0e event cmsCRC error counter for event builder 1
0x1c0f event cmsCRC error counter for event builder 2
0x1c19 word count output of event builder 0
0x1c1a word count output of event builder 1
0x1c1b word count output of event builder 2
0x1c1c SFP0 IP address register R/W (default 192.168.1.32)
0x1c1d SFP1 IP address register R/W (default 192.168.1.33)
0x1c1e SFP2 IP address register R/W (default 192.168.1.34)
0x1c1f congistion window upper limit register R/W (default 0x3fffffff)
0x1c20 stop monitor buffer overwrite on error register(default is stop on any error)
bit 31-5 not used
bit 4 if 1, stop on AMC CRC error
bit 3 if 1, stop on AMC evn, orn or bcn mismatch
bit 2 if 1, stop on AMC length error
bit 1 if 1, stop on CMS event length error
bit 0 if 1, stop on CMS CRC error

In particular, I'm interested in

0x1c1c SFP0 IP address register R/W (default 192.168.1.32)

So I fired up the help table for the amc13

[root@dhcp-10-163-105-238 amc13StandaloneMAN_2014-05-12]# bin/AMC13Tool -u
Connecting to AMC13...
T2 ip 192.168.1.188
T2 ip 192.168.1.188
T1 ip 192.168.1.189
Serial Number 33
T2 URI: ipbusudp-2.0://192.168.1.188:50001
T2 Address Table: file://map/AMC13_AddressTable_S6.xml
T1 URI: ipbusudp-2.0://192.168.1.189:50001
T1 Address Table: file://map/AMC13_AddressTable_K7.xml

Pick an action (h for menu): h

...

------- AMC13 Read/Write Commands -------
rs(v)  <add> [count]          single read from spartan (virtex) <add> to <add>+[count]
brs(v) <add> [count]          block read [count] words from spartan (virtex) <add>
frs(v) <add> [count]          fifo read spartan (virtex) <add> [count] times
ws(v)  <add> <data> [count]   single write <data> from spartan (virtex) <add> to <add>+[count]
bws(v) <add> [<data_list>]    block write [<data_list>] from spartan (virtex) <add>
fws(v) <add> [<data_list>]    fifo write [<data_list>] to spartan (virtex) <add>

...
[root@dhcp-10-163-105-238 amc13StandaloneMAN_2014-05-12]# bin/AMC13Tool -u
Connecting to AMC13...
T2 ip 192.168.1.188
T2 ip 192.168.1.188
T1 ip 192.168.1.189
Serial Number 33
T2 URI: ipbusudp-2.0://192.168.1.188:50001
T2 Address Table: file://map/AMC13_AddressTable_S6.xml
T1 URI: ipbusudp-2.0://192.168.1.189:50001
T1 Address Table: file://map/AMC13_AddressTable_K7.xml

Pick an action (h for menu): h

...

------- AMC13 Read/Write Commands -------
rs(v)  <add> [count]          single read from spartan (virtex) <add> to <add>+[count]
brs(v) <add> [count]          block read [count] words from spartan (virtex) <add>
frs(v) <add> [count]          fifo read spartan (virtex) <add> [count] times
ws(v)  <add> <data> [count]   single write <data> from spartan (virtex) <add> to <add>+[count]
bws(v) <add> [<data_list>]    block write [<data_list>] from spartan (virtex) <add>
fws(v) <add> [<data_list>]    fifo write [<data_list>] to spartan (virtex) <add>

...

Low and behold, if I read 0x1c1c on the virtex:

Pick an action (h for menu): rv 0x1c1c
Reading T1:
  00001c1c: c0a80120
Pick an action (h for menu): rv 0x1c1c
Reading T1:
  00001c1c: c0a80120

Which corresponds to:

c0 = 192
a8 = 168
01 = 1
20 = 32
c0 = 192
a8 = 168
01 = 1
20 = 32

which is our IP.

27/03/2024 20:38

I am trying to change the IP to 192.168.10.1

Pick an action (h for menu): wv 0x1c1c 0xc0a80a01
Writing to T1:
  00001c1c: c0a80a01

Pick an action (h for menu): rv 0x1c1c
Reading T1:
  00001c1c: c0a80a01
Pick an action (h for menu): wv 0x1c1c 0xc0a80a01
Writing to T1:
  00001c1c: c0a80a01

Pick an action (h for menu): rv 0x1c1c
Reading T1:
  00001c1c: c0a80a01

since

192 = c0
168 = a8
10  = 0a
1   = 01
192 = c0
168 = a8
10  = 0a
1   = 01

I then changed enp1s0f1 to be on the 192.168.10.xxx subnet with IP 192.168.1.2. It didn't really seem to work:

[root@dhcp-10-163-105-238 amc13StandaloneMAN_2014-05-12]# ping 192.168.10.1
PING 192.168.10.1 (192.168.10.1) 56(84) bytes of data.
^C
--- 192.168.10.1 ping statistics ---
2 packets transmitted, 0 received, 100% packet loss, time 999ms

[root@dhcp-10-163-105-238 amc13StandaloneMAN_2014-05-12]# ifdown enp5s0
Device 'enp5s0' successfully disconnected.
[root@dhcp-10-163-105-238 amc13StandaloneMAN_2014-05-12]# ping 192.168.10.1
PING 192.168.10.1 (192.168.10.1) 56(84) bytes of data.
^C
--- 192.168.10.1 ping statistics ---
1 packets transmitted, 0 received, 100% packet loss, time 0ms

[root@dhcp-10-163-105-238 amc13StandaloneMAN_2014-05-12]#
[root@dhcp-10-163-105-238 amc13StandaloneMAN_2014-05-12]# ping 192.168.10.1
PING 192.168.10.1 (192.168.10.1) 56(84) bytes of data.
^C
--- 192.168.10.1 ping statistics ---
2 packets transmitted, 0 received, 100% packet loss, time 999ms

[root@dhcp-10-163-105-238 amc13StandaloneMAN_2014-05-12]# ifdown enp5s0
Device 'enp5s0' successfully disconnected.
[root@dhcp-10-163-105-238 amc13StandaloneMAN_2014-05-12]# ping 192.168.10.1
PING 192.168.10.1 (192.168.10.1) 56(84) bytes of data.
^C
--- 192.168.10.1 ping statistics ---
1 packets transmitted, 0 received, 100% packet loss, time 0ms

[root@dhcp-10-163-105-238 amc13StandaloneMAN_2014-05-12]#

I can successfully change the IP to something else on the 192.168.1.xxx subnet though:

[root@dhcp-10-163-105-238 amc13StandaloneMAN_2014-05-12]# bin/AMC13Tool -u
Connecting to AMC13...
T2 ip 192.168.1.188
T2 ip 192.168.1.188
T1 ip 192.168.1.189
Serial Number 33
T2 URI: ipbusudp-2.0://192.168.1.188:50001
T2 Address Table: file://map/AMC13_AddressTable_S6.xml
T1 URI: ipbusudp-2.0://192.168.1.189:50001
T1 Address Table: file://map/AMC13_AddressTable_K7.xml

Pick an action (h for menu): wv 0x1c1c 0xc0a8011e
Writing to T1:
  00001c1c: c0a8011e

Pick an action (h for menu): exit
Invalid command

Pick an action (h for menu): ^C
[root@dhcp-10-163-105-238 amc13StandaloneMAN_2014-05-12]# ifdown enp5s0
Device 'enp5s0' successfully disconnected.
[root@dhcp-10-163-105-238 amc13StandaloneMAN_2014-05-12]# ping 192.168.1.30
PING 192.168.1.30 (192.168.1.30) 56(84) bytes of data.
64 bytes from 192.168.1.30: icmp_seq=1 ttl=64 time=0.155 ms
64 bytes from 192.168.1.30: icmp_seq=2 ttl=64 time=0.089 ms
[root@dhcp-10-163-105-238 amc13StandaloneMAN_2014-05-12]# bin/AMC13Tool -u
Connecting to AMC13...
T2 ip 192.168.1.188
T2 ip 192.168.1.188
T1 ip 192.168.1.189
Serial Number 33
T2 URI: ipbusudp-2.0://192.168.1.188:50001
T2 Address Table: file://map/AMC13_AddressTable_S6.xml
T1 URI: ipbusudp-2.0://192.168.1.189:50001
T1 Address Table: file://map/AMC13_AddressTable_K7.xml

Pick an action (h for menu): wv 0x1c1c 0xc0a8011e
Writing to T1:
  00001c1c: c0a8011e

Pick an action (h for menu): exit
Invalid command

Pick an action (h for menu): ^C
[root@dhcp-10-163-105-238 amc13StandaloneMAN_2014-05-12]# ifdown enp5s0
Device 'enp5s0' successfully disconnected.
[root@dhcp-10-163-105-238 amc13StandaloneMAN_2014-05-12]# ping 192.168.1.30
PING 192.168.1.30 (192.168.1.30) 56(84) bytes of data.
64 bytes from 192.168.1.30: icmp_seq=1 ttl=64 time=0.155 ms
64 bytes from 192.168.1.30: icmp_seq=2 ttl=64 time=0.089 ms

27/03/2024 20:53

My theory was that the changing the IP to 192.168.10.1 did not work because the 10GbE NIC needs to be on the same subnet as the virtex (which is 192.168.1.188 (or 189, I forgot)). But it looks like on the UW DAQ, they have the 10GbE NIC on 192.168.27.1 while the MCH is on 192.168.7.1.

My only idea of how they accomplished this is they changed the IPs of T1 and T2 to be on the 192.168.27.xxx subet, so I tried that:

First, while I was still able to talk to T1 and T2 through the 1GbE NIC, I changed the IP of the 10GbE NIC on the AMC13

[root@dhcp-10-163-105-238 amc13StandaloneMAN_2014-05-12]# bin/AMC13Tool -u
Connecting to AMC13...
T2 ip 192.168.1.188
T2 ip 192.168.1.188
T1 ip 192.168.1.189
Serial Number 33
T2 URI: ipbusudp-2.0://192.168.1.188:50001
T2 Address Table: file://map/AMC13_AddressTable_S6.xml
T1 URI: ipbusudp-2.0://192.168.1.189:50001
T1 Address Table: file://map/AMC13_AddressTable_K7.xml

Pick an action (h for menu): wv 0x1c1c 0xc0a81b01
Writing to T1:
  00001c1c: c0a81b01
[root@dhcp-10-163-105-238 amc13StandaloneMAN_2014-05-12]# bin/AMC13Tool -u
Connecting to AMC13...
T2 ip 192.168.1.188
T2 ip 192.168.1.188
T1 ip 192.168.1.189
Serial Number 33
T2 URI: ipbusudp-2.0://192.168.1.188:50001
T2 Address Table: file://map/AMC13_AddressTable_S6.xml
T1 URI: ipbusudp-2.0://192.168.1.189:50001
T1 Address Table: file://map/AMC13_AddressTable_K7.xml

Pick an action (h for menu): wv 0x1c1c 0xc0a81b01
Writing to T1:
  00001c1c: c0a81b01

This corresponds to

192 = c0
168 = a8
27  = 1b
1   = 01
192 = c0
168 = a8
27  = 1b
1   = 01

Then I changed systemVars.py to change the network base:

[root@dhcp-10-163-105-238 amc13Config]# cat systemVars.py

#File to specify what the default varaibles addresses are used in your system

#Default IP address for commercial MCH module
# our NAT MCH address
DEFAULT_HOST_IP="192.168.1.41"
# our Vadatech MCH address
# DEFAULT_HOST_IP="192.168.1.2"

#Default AMC13 slot number
DEFAULT_AMC13_SLOT=13

#Location of 'config_tools'. This should never need to be changed
DEFAULT_CONFIG_DIR="./config_tools"

#Network base for your uTCA crate's AMC modules
NETWORK_BASE="192.168.27"
[root@dhcp-10-163-105-238 amc13Config]# cat systemVars.py

#File to specify what the default varaibles addresses are used in your system

#Default IP address for commercial MCH module
# our NAT MCH address
DEFAULT_HOST_IP="192.168.1.41"
# our Vadatech MCH address
# DEFAULT_HOST_IP="192.168.1.2"

#Default AMC13 slot number
DEFAULT_AMC13_SLOT=13

#Location of 'config_tools'. This should never need to be changed
DEFAULT_CONFIG_DIR="./config_tools"

#Network base for your uTCA crate's AMC modules
NETWORK_BASE="192.168.27"

I then ran ./applyConfig -n 33

[root@dhcp-10-163-105-238 amc13Config]# ./applyConfig.py -n 33
Applying IP addresses to board in slot 13 from host 192.168.1.41
Setting IP addresses using SN 33
 ipmitool -H 192.168.1.41 -U '' -P '' -m 0x20 -T 0x82 -b 7 -t 0xa4 raw 0x32 0x33 0 0 0 11 0x0d 0xff 0xff 0xff 0x00 0xc0 0xa8 0x1b 0xbc 0x00 0x00

 ipmitool -H 192.168.1.41 -U '' -P '' -m 0x20 -T 0x82 -b 7 -t 0xa4 raw 0x32 0x33 1 0 0 11 0x0d 0xff 0xff 0xff 0x00 0xc0 0xa8 0x1b 0xbd 0x00 0x00

[root@dhcp-10-163-105-238 amc13Config]# ./applyConfig.py -n 33
Applying IP addresses to board in slot 13 from host 192.168.1.41
Setting IP addresses using SN 33
 ipmitool -H 192.168.1.41 -U '' -P '' -m 0x20 -T 0x82 -b 7 -t 0xa4 raw 0x32 0x33 0 0 0 11 0x0d 0xff 0xff 0xff 0x00 0xc0 0xa8 0x1b 0xbc 0x00 0x00

 ipmitool -H 192.168.1.41 -U '' -P '' -m 0x20 -T 0x82 -b 7 -t 0xa4 raw 0x32 0x33 1 0 0 11 0x0d 0xff 0xff 0xff 0x00 0xc0 0xa8 0x1b 0xbd 0x00 0x00

We see the new IPs are set to:
0xc0 0xa8 0x1b 0xbc --> 192.168.27.188
0xc0 0xa8 0x1b 0xbd --> 102.168.27.189

Then I check that I can no longer ping T1 and T2 (I shouldn't be able to)

[root@dhcp-10-163-105-238 amc13Config]# ping 192.168.1.188
PING 192.168.1.188 (192.168.1.188) 56(84) bytes of data.
^C
--- 192.168.1.188 ping statistics ---
2 packets transmitted, 0 received, 100% packet loss, time 999ms

[root@dhcp-10-163-105-238 amc13Config]# ping 192.168.1.189
PING 192.168.1.189 (192.168.1.189) 56(84) bytes of data.
^C
--- 192.168.1.189 ping statistics ---
1 packets transmitted, 0 received, 100% packet loss, time 0ms
[root@dhcp-10-163-105-238 amc13Config]# ping 192.168.1.188
PING 192.168.1.188 (192.168.1.188) 56(84) bytes of data.
^C
--- 192.168.1.188 ping statistics ---
2 packets transmitted, 0 received, 100% packet loss, time 999ms

[root@dhcp-10-163-105-238 amc13Config]# ping 192.168.1.189
PING 192.168.1.189 (192.168.1.189) 56(84) bytes of data.
^C
--- 192.168.1.189 ping statistics ---
1 packets transmitted, 0 received, 100% packet loss, time 0ms

I then change enp1s0f1 to be on the 192.168.27.xxx subnet:

enp1s0f1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 9000
        inet 192.168.27.21  netmask 255.255.255.0  broadcast 192.168.27.255
        inet6 fe80::232a:6445:25a6:4e16  prefixlen 64  scopeid 0x20<link>
        ether b4:b5:2f:a4:e7:fc  txqueuelen 1000  (Ethernet)
        RX packets 37  bytes 2790 (2.7 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 841  bytes 102979 (100.5 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
enp1s0f1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 9000
        inet 192.168.27.21  netmask 255.255.255.0  broadcast 192.168.27.255
        inet6 fe80::232a:6445:25a6:4e16  prefixlen 64  scopeid 0x20<link>
        ether b4:b5:2f:a4:e7:fc  txqueuelen 1000  (Ethernet)
        RX packets 37  bytes 2790 (2.7 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 841  bytes 102979 (100.5 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

And try to ping:

[root@dhcp-10-163-105-238 network-scripts]# ping 192.168.27.1
PING 192.168.27.1 (192.168.27.1) 56(84) bytes of data.
From 192.168.27.21 icmp_seq=1 Destination Host Unreachable
From 192.168.27.21 icmp_seq=2 Destination Host Unreachable
From 192.168.27.21 icmp_seq=3 Destination Host Unreachable
From 192.168.27.21 icmp_seq=4 Destination Host Unreachable
[root@dhcp-10-163-105-238 network-scripts]# ping 192.168.27.1
PING 192.168.27.1 (192.168.27.1) 56(84) bytes of data.
From 192.168.27.21 icmp_seq=1 Destination Host Unreachable
From 192.168.27.21 icmp_seq=2 Destination Host Unreachable
From 192.168.27.21 icmp_seq=3 Destination Host Unreachable
From 192.168.27.21 icmp_seq=4 Destination Host Unreachable

But no luck.

On second though, this procedure makes no sense anyway. The frontend needs to see T1 and T2, which means they need to be on the same subnet as the 1GbE NIC. I think UW accomplishes this by putting the 1GbE NIC on the 192.168.xxx.xxx subnet. I can't see exactly what they did right now because it's in use, but this doesn't seem right because I can't ping what I think T1 and T2 should be:

[root@cenpa-pioneer j.carlton]# ping 192.168.27.188
PING 192.168.27.188 (192.168.27.188) 56(84) bytes of data.
^C
--- 192.168.27.188 ping statistics ---
2 packets transmitted, 0 received, 100% packet loss, time 999ms

[root@cenpa-pioneer j.carlton]# ping 192.168.27.189
PING 192.168.27.189 (192.168.27.189) 56(84) bytes of data.
^C
--- 192.168.27.189 ping statistics ---
1 packets transmitted, 0 received, 100% packet loss, time 0ms
[root@cenpa-pioneer j.carlton]# ping 192.168.27.188
PING 192.168.27.188 (192.168.27.188) 56(84) bytes of data.
^C
--- 192.168.27.188 ping statistics ---
2 packets transmitted, 0 received, 100% packet loss, time 999ms

[root@cenpa-pioneer j.carlton]# ping 192.168.27.189
PING 192.168.27.189 (192.168.27.189) 56(84) bytes of data.
^C
--- 192.168.27.189 ping statistics ---
1 packets transmitted, 0 received, 100% packet loss, time 0ms

But it's possible they have different IPs (not 188 and 189)

25/03/2024 06:12

Hi Tim,
the easiest way I think is to install it from CERN repository. See details below:

Name : cactus-amc13
Arch : noarch
Version : 1.2.2
Release : 20170214
Size : 0.0
Repo : installed
From repo : cactus_amc13_noarch
Summary : Wrapper meta RPM for cactus-amc13 YUM group
URL : https://git.cern.ch/web/cms-daq-sysadmins-public.git
License : UNKNOWN
Description : Meta RPM that pulls all the cactus-amc13 Yum group required RPMs

This is a printout from one of my machines in CERN that has amc13 tools installed. The command that generated this printout is:

sudo yum info cactus-amc13

I did not install it myself though, so don't have step-by-step instructions. Let me know if you need further help with this.
Alex.

Unsurprisngly, just trying to see if we have the RPM shows we don't:

[root@dhcp-10-163-105-238 tools]# sudo yum info cactus-amc13
Loaded plugins: fastestmirror, langpacks
Loading mirror speeds from cached hostfile
 * base: mirror.us-midwest-1.nexcess.net
 * centos-sclo-rh: mirror.us.oneandone.net
 * centos-sclo-sclo: mirror.team-cymru.com
 * epel: mirrors.lug.mtu.edu
 * extras: coresite.mm.fcix.net
 * updates: mirror.pit.teraswitch.com
Error: No matching Packages to list
[root@dhcp-10-163-105-238 tools]# sudo yum install cactus-amc13
Loaded plugins: fastestmirror, langpacks
Loading mirror speeds from cached hostfile
 * base: mirror.us-midwest-1.nexcess.net
 * centos-sclo-rh: mirror.us.oneandone.net
 * centos-sclo-sclo: mirror.team-cymru.com
 * epel: mirrors.lug.mtu.edu
 * extras: coresite.mm.fcix.net
 * updates: mirror.pit.teraswitch.com
No package cactus-amc13 available.
Error: Nothing to do
[root@dhcp-10-163-105-238 tools]#
[root@dhcp-10-163-105-238 tools]# sudo yum info cactus-amc13
Loaded plugins: fastestmirror, langpacks
Loading mirror speeds from cached hostfile
 * base: mirror.us-midwest-1.nexcess.net
 * centos-sclo-rh: mirror.us.oneandone.net
 * centos-sclo-sclo: mirror.team-cymru.com
 * epel: mirrors.lug.mtu.edu
 * extras: coresite.mm.fcix.net
 * updates: mirror.pit.teraswitch.com
Error: No matching Packages to list
[root@dhcp-10-163-105-238 tools]# sudo yum install cactus-amc13
Loaded plugins: fastestmirror, langpacks
Loading mirror speeds from cached hostfile
 * base: mirror.us-midwest-1.nexcess.net
 * centos-sclo-rh: mirror.us.oneandone.net
 * centos-sclo-sclo: mirror.team-cymru.com
 * epel: mirrors.lug.mtu.edu
 * extras: coresite.mm.fcix.net
 * updates: mirror.pit.teraswitch.com
No package cactus-amc13 available.
Error: Nothing to do
[root@dhcp-10-163-105-238 tools]#

I tried checking out the link Alex gave from when he ran sudo yum info cactus-amc13 on his machine:
https://git.cern.ch/web/cms-daq-sysadmins-public.git
But I'm blocked by a CERN login page.

25/03/2024 06:20

I found this on a google search:
https://gitlab.cern.ch/cms-cactus/boards/amc13
But I can't seem to clone it, I'm suspecting similar permission issues.

[root@dhcp-10-163-105-238 tools]# git clone --recurse-submodules https://gitlab.cern.ch/cms-cactus/boards/amc13
Cloning into 'amc13'...
error: RPC failed; result=22, HTTP code = 422
fatal: The remote end hung up unexpectedly
[root@dhcp-10-163-105-238 tools]#
[root@dhcp-10-163-105-238 tools]# git clone --recurse-submodules https://gitlab.cern.ch/cms-cactus/boards/amc13
Cloning into 'amc13'...
error: RPC failed; result=22, HTTP code = 422
fatal: The remote end hung up unexpectedly
[root@dhcp-10-163-105-238 tools]#

Trying the ssh variant of the clone, the command hangs:

[root@dhcp-10-163-105-238 tools]# git clone --recurse-submodules git@gitlab.cern.ch:cms-cactus/boards/amc13
Cloning into 'amc13'...

[root@dhcp-10-163-105-238 tools]# git clone --recurse-submodules git@gitlab.cern.ch:cms-cactus/boards/amc13
Cloning into 'amc13'...

25/03/2024 06:22

By just looking at the file structure on https://gitlab.cern.ch/cms-cactus/boards/amc13, I noticed it look similar to what we have in /home/installation_testing/packages/experiment/lxedaq/amc13/amc13_v1_2_18.
And it turns out AMC13Tool2 is indeed in here:

[root@dhcp-10-163-105-238 common]# ls
AMC13BenchTest.cxx  AMC13ToolFlash.cxx  CLIHelper.cc    Launcher.cc           Launcher_commands_control.cc  Launcher_commands_io.cc      Module.cc
AMC13Tool2.cxx      CLI.cc              LaTeXprint.cxx  Launcher_commands.cc  Launcher_commands_flash.cc    Launcher_commands_status.cc
[root@dhcp-10-163-105-238 common]# pwd
/home/installation_testing/packages/experiment/lxedaq/amc13/amc13_v1_2_18/tools/src/common
[root@dhcp-10-163-105-238 common]#
[root@dhcp-10-163-105-238 common]# ls
AMC13BenchTest.cxx  AMC13ToolFlash.cxx  CLIHelper.cc    Launcher.cc           Launcher_commands_control.cc  Launcher_commands_io.cc      Module.cc
AMC13Tool2.cxx      CLI.cc              LaTeXprint.cxx  Launcher_commands.cc  Launcher_commands_flash.cc    Launcher_commands_status.cc
[root@dhcp-10-163-105-238 common]# pwd
/home/installation_testing/packages/experiment/lxedaq/amc13/amc13_v1_2_18/tools/src/common
[root@dhcp-10-163-105-238 common]#

So I tried making it (first I have to setup the environment):

  1. cd /home/installation_testing/packages/experiment/lxedaq/environment_setup/
  2. source ./setup_environment.sh
  3. export CPLUS_INCLUDE_PATH=$CPLUS_INCLUDE_PATH:/home/backup_installation_testing/packages/boost-1.53.0/include (adds boost to C++ include path)

After doing that, I try making:

  1. cd /home/installation_testing/packages/experiment/lxedaq/amc13/amc13_v1_2_18/tools
  2. make

But get these errors:

[root@dhcp-10-163-105-238 tools]# make
Using AMC13_ROOT=/home/installation_testing/packages/experiment/lxedaq/amc13/amc13_v1_2_18/tools/../
Using BUILD_HOME=/home/installation_testing/packages/experiment/lxedaq/amc13/amc13_v1_2_18/tools/../
OS Detected: centos7
mkdir -p {lib,obj}
g++ -g -O3 -rdynamic -Wall -MMD -MP -fPIC -std=c++11 -Iinclude -I../amc13//include -I/opt/cactus/include  -c src/common/Launcher_commands.cc -o obj/Launcher_commands.o
In file included from ../amc13//include/amc13/AMC13.hh:9,
                 from include/amc13/Launcher.hh:11,
                 from src/common/Launcher_commands.cc:1:
../amc13//include/amc13/Status.hh:100:31: error: ‘std::unordered_map’ has not been declared
     std::string ParseRow(std::unordered_map<std::string,std::string> & parameters,
                               ^~~~~~~~~~~~~
../amc13//include/amc13/Status.hh:100:44: error: expected ‘,’ or ‘...’ before ‘<’ token
     std::string ParseRow(std::unordered_map<std::string,std::string> & parameters,
                                            ^
../amc13//include/amc13/Status.hh:102:31: error: ‘std::unordered_map’ has not been declared
     std::string ParseCol(std::unordered_map<std::string,std::string> & parameters,
                               ^~~~~~~~~~~~~
../amc13//include/amc13/Status.hh:102:44: error: expected ‘,’ or ‘...’ before ‘<’ token
     std::string ParseCol(std::unordered_map<std::string,std::string> & parameters,
                                            ^
make: *** [Makefile:100: obj/Launcher_commands.o] Error 1
[root@dhcp-10-163-105-238 tools]#
[root@dhcp-10-163-105-238 tools]# make
Using AMC13_ROOT=/home/installation_testing/packages/experiment/lxedaq/amc13/amc13_v1_2_18/tools/../
Using BUILD_HOME=/home/installation_testing/packages/experiment/lxedaq/amc13/amc13_v1_2_18/tools/../
OS Detected: centos7
mkdir -p {lib,obj}
g++ -g -O3 -rdynamic -Wall -MMD -MP -fPIC -std=c++11 -Iinclude -I../amc13//include -I/opt/cactus/include  -c src/common/Launcher_commands.cc -o obj/Launcher_commands.o
In file included from ../amc13//include/amc13/AMC13.hh:9,
                 from include/amc13/Launcher.hh:11,
                 from src/common/Launcher_commands.cc:1:
../amc13//include/amc13/Status.hh:100:31: error: ‘std::unordered_map’ has not been declared
     std::string ParseRow(std::unordered_map<std::string,std::string> & parameters,
                               ^~~~~~~~~~~~~
../amc13//include/amc13/Status.hh:100:44: error: expected ‘,’ or ‘...’ before ‘<’ token
     std::string ParseRow(std::unordered_map<std::string,std::string> & parameters,
                                            ^
../amc13//include/amc13/Status.hh:102:31: error: ‘std::unordered_map’ has not been declared
     std::string ParseCol(std::unordered_map<std::string,std::string> & parameters,
                               ^~~~~~~~~~~~~
../amc13//include/amc13/Status.hh:102:44: error: expected ‘,’ or ‘...’ before ‘<’ token
     std::string ParseCol(std::unordered_map<std::string,std::string> & parameters,
                                            ^
make: *** [Makefile:100: obj/Launcher_commands.o] Error 1
[root@dhcp-10-163-105-238 tools]#

This error seems very dumb to me. The project includes boost unordered maps, but uses standard c++ library unordered maps here (???). Unless there's some hidden C++ magic here, this just seems like faulty code. The "newer" code (see here: https://gitlab.cern.ch/cms-cactus/boards/amc13/-/blob/master/amc13/include/amc13/Status.hh?ref_type=heads) doesn't even used unordered maps. Anyways, I tried fixing it by adding

#include <unordered_map>
#include <unordered_map>

to Status.hh. That got past this error but we quickly reached another:

[root@dhcp-10-163-105-238 tools]# make clean
Using AMC13_ROOT=/home/installation_testing/packages/experiment/lxedaq/amc13/amc13_v1_2_18/tools/../
Using BUILD_HOME=/home/installation_testing/packages/experiment/lxedaq/amc13/amc13_v1_2_18/tools/../
OS Detected: centos7
rm -rf obj
rm -rf bin
rm -rf lib
[root@dhcp-10-163-105-238 tools]# make
Using AMC13_ROOT=/home/installation_testing/packages/experiment/lxedaq/amc13/amc13_v1_2_18/tools/../
Using BUILD_HOME=/home/installation_testing/packages/experiment/lxedaq/amc13/amc13_v1_2_18/tools/../
OS Detected: centos7
mkdir -p {lib,obj}
g++ -g -O3 -rdynamic -Wall -MMD -MP -fPIC -std=c++11 -Iinclude -I../amc13//include -I/home/installation_testing/packages/cactus/include  -c src/common/Launcher_commands.cc -o obj/Launcher_commands.o
mkdir -p {lib,obj}
g++ -g -O3 -rdynamic -Wall -MMD -MP -fPIC -std=c++11 -Iinclude -I../amc13//include -I/home/installation_testing/packages/cactus/include  -c src/common/CLI.cc -o obj/CLI.o
mkdir -p {lib,obj}
g++ -g -O3 -rdynamic -Wall -MMD -MP -fPIC -std=c++11 -Iinclude -I../amc13//include -I/home/installation_testing/packages/cactus/include  -c src/common/Module.cc -o obj/Module.o
mkdir -p {lib,obj}
g++ -g -O3 -rdynamic -Wall -MMD -MP -fPIC -std=c++11 -Iinclude -I../amc13//include -I/home/installation_testing/packages/cactus/include  -c src/common/Launcher_commands_io.cc -o obj/Launcher_commands_io.o
mkdir -p {lib,obj}
g++ -g -O3 -rdynamic -Wall -MMD -MP -fPIC -std=c++11 -Iinclude -I../amc13//include -I/home/installation_testing/packages/cactus/include  -c src/common/Launcher_commands_status.cc -o obj/Launcher_commands_status.o
src/common/Launcher_commands_status.cc: In function ‘void amc13::statusTableAutoCompleteHelper(const uhal::Node&, std::vector<std::basic_string<char> >&, const string&)’:
src/common/Launcher_commands_status.cc:242:85: error: conversion from ‘const boost::unordered::unordered_map<std::basic_string<char>, std::basic_string<char> >’ to non-scalar type ‘std::unordered_map<std::basic_string<char>, std::basic_string<char> >’ requested
       std::unordered_map<std::string,std::string> parameters = itNode->getParameters();
                                                                ~~~~~~~~~~~~~~~~~~~~~^~
make: *** [Makefile:100: obj/Launcher_commands_status.o] Error 1
[root@dhcp-10-163-105-238 tools]# make clean
Using AMC13_ROOT=/home/installation_testing/packages/experiment/lxedaq/amc13/amc13_v1_2_18/tools/../
Using BUILD_HOME=/home/installation_testing/packages/experiment/lxedaq/amc13/amc13_v1_2_18/tools/../
OS Detected: centos7
rm -rf obj
rm -rf bin
rm -rf lib
[root@dhcp-10-163-105-238 tools]# make
Using AMC13_ROOT=/home/installation_testing/packages/experiment/lxedaq/amc13/amc13_v1_2_18/tools/../
Using BUILD_HOME=/home/installation_testing/packages/experiment/lxedaq/amc13/amc13_v1_2_18/tools/../
OS Detected: centos7
mkdir -p {lib,obj}
g++ -g -O3 -rdynamic -Wall -MMD -MP -fPIC -std=c++11 -Iinclude -I../amc13//include -I/home/installation_testing/packages/cactus/include  -c src/common/Launcher_commands.cc -o obj/Launcher_commands.o
mkdir -p {lib,obj}
g++ -g -O3 -rdynamic -Wall -MMD -MP -fPIC -std=c++11 -Iinclude -I../amc13//include -I/home/installation_testing/packages/cactus/include  -c src/common/CLI.cc -o obj/CLI.o
mkdir -p {lib,obj}
g++ -g -O3 -rdynamic -Wall -MMD -MP -fPIC -std=c++11 -Iinclude -I../amc13//include -I/home/installation_testing/packages/cactus/include  -c src/common/Module.cc -o obj/Module.o
mkdir -p {lib,obj}
g++ -g -O3 -rdynamic -Wall -MMD -MP -fPIC -std=c++11 -Iinclude -I../amc13//include -I/home/installation_testing/packages/cactus/include  -c src/common/Launcher_commands_io.cc -o obj/Launcher_commands_io.o
mkdir -p {lib,obj}
g++ -g -O3 -rdynamic -Wall -MMD -MP -fPIC -std=c++11 -Iinclude -I../amc13//include -I/home/installation_testing/packages/cactus/include  -c src/common/Launcher_commands_status.cc -o obj/Launcher_commands_status.o
src/common/Launcher_commands_status.cc: In function ‘void amc13::statusTableAutoCompleteHelper(const uhal::Node&, std::vector<std::basic_string<char> >&, const string&)’:
src/common/Launcher_commands_status.cc:242:85: error: conversion from ‘const boost::unordered::unordered_map<std::basic_string<char>, std::basic_string<char> >’ to non-scalar type ‘std::unordered_map<std::basic_string<char>, std::basic_string<char> >’ requested
       std::unordered_map<std::string,std::string> parameters = itNode->getParameters();
                                                                ~~~~~~~~~~~~~~~~~~~~~^~
make: *** [Makefile:100: obj/Launcher_commands_status.o] Error 1

This error has to do with the fact that itNode is returning a boost unordered map and C++ doesn't know how to convert that to a standard C++ library unordered map. So I tried changing the relevant to line to use boost's unordered map:

boost::unordered_map<std::string,std::string> parameters = itNode->getParameters();
boost::unordered_map<std::string,std::string> parameters = itNode->getParameters();

That seemed to cause a lot more trouble "down the road" in the code by the errors after:

root@dhcp-10-163-105-238 tools]# make
Using AMC13_ROOT=/home/installation_testing/packages/experiment/lxedaq/amc13/amc13_v1_2_18/tools/../
Using BUILD_HOME=/home/installation_testing/packages/experiment/lxedaq/amc13/amc13_v1_2_18/tools/../
OS Detected: centos7
mkdir -p {lib,obj}
g++ -g -O3 -rdynamic -Wall -MMD -MP -fPIC -std=c++11 -Iinclude -I../amc13//include -I/home/installation_testing/packages/cactus/include  -c src/common/Launcher_commands_status.cc -o obj/Launcher_commands_status.o
src/common/Launcher_commands_status.cc: In function ‘void amc13::statusTableAutoCompleteHelper(const uhal::Node&, std::vector<std::basic_string<char> >&, const string&)’:
src/common/Launcher_commands_status.cc:244:40: error: no match for ‘operator=’ (operand types are ‘std::unordered_map<std::basic_string<char>, std::basic_string<char> >::iterator’ {aka ‘std::__detail::_Node_iterator<std::pair<const std::basic_string<char>, std::basic_string<char> >, false, true>’} and ‘boost::unordered::unordered_map<std::basic_string<char>, std::basic_string<char> >::iterator’ {aka ‘boost::unordered::iterator_detail::iterator<boost::unordered::detail::ptr_node<std::pair<const std::basic_string<char>, std::basic_string<char> > > >’})
       itTable = parameters.find("Table");
                                        ^
In file included from /opt/rh/devtoolset-8/root/usr/include/c++/8/bits/hashtable.h:35,
                 from /opt/rh/devtoolset-8/root/usr/include/c++/8/unordered_map:46,
                 from ../amc13//include/amc13/Status.hh:14,
                 from ../amc13//include/amc13/AMC13.hh:9,
                 from include/amc13/Launcher.hh:11,
                 from src/common/Launcher_commands_status.cc:1:
/opt/rh/devtoolset-8/root/usr/include/c++/8/bits/hashtable_policy.h:319:12: note: candidate: ‘std::__detail::_Node_iterator<std::pair<const std::basic_string<char>, std::basic_string<char> >, false, true>& std::__detail::_Node_iterator<std::pair<const std::basic_string<char>, std::basic_string<char> >, false, true>::operator=(const std::__detail::_Node_iterator<std::pair<const std::basic_string<char>, std::basic_string<char> >, false, true>&)’
     struct _Node_iterator
     
...

root@dhcp-10-163-105-238 tools]# make
Using AMC13_ROOT=/home/installation_testing/packages/experiment/lxedaq/amc13/amc13_v1_2_18/tools/../
Using BUILD_HOME=/home/installation_testing/packages/experiment/lxedaq/amc13/amc13_v1_2_18/tools/../
OS Detected: centos7
mkdir -p {lib,obj}
g++ -g -O3 -rdynamic -Wall -MMD -MP -fPIC -std=c++11 -Iinclude -I../amc13//include -I/home/installation_testing/packages/cactus/include  -c src/common/Launcher_commands_status.cc -o obj/Launcher_commands_status.o
src/common/Launcher_commands_status.cc: In function ‘void amc13::statusTableAutoCompleteHelper(const uhal::Node&, std::vector<std::basic_string<char> >&, const string&)’:
src/common/Launcher_commands_status.cc:244:40: error: no match for ‘operator=’ (operand types are ‘std::unordered_map<std::basic_string<char>, std::basic_string<char> >::iterator’ {aka ‘std::__detail::_Node_iterator<std::pair<const std::basic_string<char>, std::basic_string<char> >, false, true>’} and ‘boost::unordered::unordered_map<std::basic_string<char>, std::basic_string<char> >::iterator’ {aka ‘boost::unordered::iterator_detail::iterator<boost::unordered::detail::ptr_node<std::pair<const std::basic_string<char>, std::basic_string<char> > > >’})
       itTable = parameters.find("Table");
                                        ^
In file included from /opt/rh/devtoolset-8/root/usr/include/c++/8/bits/hashtable.h:35,
                 from /opt/rh/devtoolset-8/root/usr/include/c++/8/unordered_map:46,
                 from ../amc13//include/amc13/Status.hh:14,
                 from ../amc13//include/amc13/AMC13.hh:9,
                 from include/amc13/Launcher.hh:11,
                 from src/common/Launcher_commands_status.cc:1:
/opt/rh/devtoolset-8/root/usr/include/c++/8/bits/hashtable_policy.h:319:12: note: candidate: ‘std::__detail::_Node_iterator<std::pair<const std::basic_string<char>, std::basic_string<char> >, false, true>& std::__detail::_Node_iterator<std::pair<const std::basic_string<char>, std::basic_string<char> >, false, true>::operator=(const std::__detail::_Node_iterator<std::pair<const std::basic_string<char>, std::basic_string<char> >, false, true>&)’
     struct _Node_iterator
     
...

I'm not quite sure why the code is so broken. For one it's old. But it seems we have it in some intermediate state where the code was still being developed. Or maybe uHAL updated to use boost unordered maps for efficiency purposes.

25/03/2024 07:27

You can check the page size on a computer like this:

[root@dhcp-10-163-105-238 tools]# getconf PAGE_SIZE
4096
[root@dhcp-10-163-105-238 tools]# getconf PAGE_SIZE
4096

(this is on the 'be' computer)
I want to check this on 'fe01'. If it reads back 2048 for some reason, that can explain the cyclic nature of our 512 4-byte words.

27/03/2024 18:42

I'm currently able to talk to the AMC through 10GbE now. However, only if enp5s0 is off:

[root@dhcp-10-163-105-238 amc13StandaloneMAN_2014-05-12]# ping 192.168.1.32
PING 192.168.1.32 (192.168.1.32) 56(84) bytes of data.
^C
--- 192.168.1.32 ping statistics ---
2 packets transmitted, 0 received, 100% packet loss, time 999ms

[root@dhcp-10-163-105-238 amc13StandaloneMAN_2014-05-12]# ifdown enp5s0
Device 'enp5s0' successfully disconnected.
[root@dhcp-10-163-105-238 amc13StandaloneMAN_2014-05-12]# ping 192.168.1.32
PING 192.168.1.32 (192.168.1.32) 56(84) bytes of data.
64 bytes from 192.168.1.32: icmp_seq=1 ttl=64 time=0.161 ms
64 bytes from 192.168.1.32: icmp_seq=2 ttl=64 time=0.093 ms
64 bytes from 192.168.1.32: icmp_seq=3 ttl=64 time=0.058 ms
^C
--- 192.168.1.32 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2000ms
rtt min/avg/max/mdev = 0.058/0.104/0.161/0.042 ms
[root@dhcp-10-163-105-238 amc13StandaloneMAN_2014-05-12]#
[root@dhcp-10-163-105-238 amc13StandaloneMAN_2014-05-12]# ping 192.168.1.32
PING 192.168.1.32 (192.168.1.32) 56(84) bytes of data.
^C
--- 192.168.1.32 ping statistics ---
2 packets transmitted, 0 received, 100% packet loss, time 999ms

[root@dhcp-10-163-105-238 amc13StandaloneMAN_2014-05-12]# ifdown enp5s0
Device 'enp5s0' successfully disconnected.
[root@dhcp-10-163-105-238 amc13StandaloneMAN_2014-05-12]# ping 192.168.1.32
PING 192.168.1.32 (192.168.1.32) 56(84) bytes of data.
64 bytes from 192.168.1.32: icmp_seq=1 ttl=64 time=0.161 ms
64 bytes from 192.168.1.32: icmp_seq=2 ttl=64 time=0.093 ms
64 bytes from 192.168.1.32: icmp_seq=3 ttl=64 time=0.058 ms
^C
--- 192.168.1.32 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2000ms
rtt min/avg/max/mdev = 0.058/0.104/0.161/0.042 ms
[root@dhcp-10-163-105-238 amc13StandaloneMAN_2014-05-12]#

To accomplish this I did a few things:

Set up enp1s0f1 port:

  1. vi ifcfg-enp1s0f1
  2. Change active lines to below
#
# Connect to AMC
#
HWADDR=b4:b5:2f:a4:e7:fc
TYPE=Ethernet
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=none
IPADDR=192.168.1.21
PREFIX=24
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
UUID=f1d52da3-687b-3215-a2c0-60c11d0fd3bf
ONBOOT=yes
AUTOCONNECT_PRIORITY=-999
MTU=9000
DEVICE=enp1s0f1
NAME=enp1s0f1
#
# Connect to AMC
#
HWADDR=b4:b5:2f:a4:e7:fc
TYPE=Ethernet
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=none
IPADDR=192.168.1.21
PREFIX=24
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
UUID=f1d52da3-687b-3215-a2c0-60c11d0fd3bf
ONBOOT=yes
AUTOCONNECT_PRIORITY=-999
MTU=9000
DEVICE=enp1s0f1
NAME=enp1s0f1

This is more or less copied from the UW setup. I had to change MTU to 9000, anything higher seemed to default the port back to an MTU of 1500.

Run AMC13Tool:

Previously in my notes we built AMC13Tool. To run it, some minor changes need to be made:

  1. replace all "." characters in /home/installation_testing/packages/experiment/lxedaq/amc13/amc13StandaloneMAN_2014-05-12/map/AMC13_AddressTable_V6.xml with something else. Tim chose "dot" so that's what I went with. I just used VSC select all tool and made sure to chang ehte xml version at the top back to "1.0".
  2. Apply environment variables:
    1. cd /home/installation_testing/packages/experiment/gm2daq/environment_setup/
    2. source ./setup_environment.sh
    3. export LD_LIBRARY_PATH=/home/installation_testing/packages/experiment/lxedaq/amc13/amc13StandaloneMAN_2014-05-12/lib:$LD_LIBRARY_PATH
    4. cd /home/installation_testing/packages/experiment/lxedaq/amc13/amc13StandaloneMAN_2014-05-12
    5. source uhalEnv.sh
  3. Run the AMC13Tool from directory /home/installation_testing/packages/experiment/lxedaq/amc13/amc13StandaloneMAN_2014-05-12

    1. bin/AMC13Tool -u
      should see output like this:

      [root@dhcp-10-163-105-238 amc13StandaloneMAN_2014-05-12]# bin/AMC13Tool -     u
Connecting to AMC13...
T2 ip 192.168.1.188
T2 ip 192.168.1.188
T1 ip 192.168.1.189
Serial Number 33
T2 URI: ipbusudp-2.0://192.168.1.188:50001
T2 Address Table: file://map/AMC13_AddressTable_S6.xml
T1 URI: ipbusudp-2.0://192.168.1.189:50001
T1 Address Table: file://map/AMC13_AddressTable_K7.xml

Pick an action (h for menu):
      [root@dhcp-10-163-105-238 amc13StandaloneMAN_2014-05-12]# bin/AMC13Tool -     u
Connecting to AMC13...
T2 ip 192.168.1.188
T2 ip 192.168.1.188
T1 ip 192.168.1.189
Serial Number 33
T2 URI: ipbusudp-2.0://192.168.1.188:50001
T2 Address Table: file://map/AMC13_AddressTable_S6.xml
T1 URI: ipbusudp-2.0://192.168.1.189:50001
T1 Address Table: file://map/AMC13_AddressTable_K7.xml

Pick an action (h for menu):

      Note: at first it complained it couldn't find AMC13 like below:

      [root@dhcp-10-163-105-238 amc13StandaloneMAN_2014-05-12]# bin/AMC13Tool -u
Connecting to AMC13...
T2 ip 192.168.1.188
T2 ip 192.168.1.188
T2 ip 192.168.1.188
T2 ip 192.168.1.188
T1 ip 192.168.1.189
T1 ip 192.168.1.189
Serial Number 33
T2 URI: ipbusudp-1.3://192.168.1.188:50001
T2 Address Table: file://map/AMC13_AddressTable_S6.xml
T1 URI: ipbusudp-1.3://192.168.1.189:50001
T1 Address Table: file://map/AMC13_AddressTable_V6.xml

***WARNING! AMC13 NOT FOUND AT THIS LOCATION!***
      [root@dhcp-10-163-105-238 amc13StandaloneMAN_2014-05-12]# bin/AMC13Tool -u
Connecting to AMC13...
T2 ip 192.168.1.188
T2 ip 192.168.1.188
T2 ip 192.168.1.188
T2 ip 192.168.1.188
T1 ip 192.168.1.189
T1 ip 192.168.1.189
Serial Number 33
T2 URI: ipbusudp-1.3://192.168.1.188:50001
T2 Address Table: file://map/AMC13_AddressTable_S6.xml
T1 URI: ipbusudp-1.3://192.168.1.189:50001
T1 Address Table: file://map/AMC13_AddressTable_V6.xml

***WARNING! AMC13 NOT FOUND AT THIS LOCATION!***

      I had to make sure port enp5s0 (our MCH 1GbE connection) was up and had priority over enp1s0f1 (our AMC13 10GbE connection) on our network:

      [root@dhcp-10-163-105-238 amc13StandaloneMAN_2014-05-12]# ip route show
default via 10.163.105.1 dev eno1 proto dhcp metric 101
10.163.105.0/24 dev eno1 proto kernel scope link src 10.163.105.238 metric 101
192.168.1.0/24 dev enp5s0 proto kernel scope link src 192.168.1.100 metric 105
192.168.1.0/24 dev enp1s0f1 proto kernel scope link src 192.168.1.21 metric 106
192.168.122.0/24 dev virbr0 proto kernel scope link src 192.168.122.1
      [root@dhcp-10-163-105-238 amc13StandaloneMAN_2014-05-12]# ip route show
default via 10.163.105.1 dev eno1 proto dhcp metric 101
10.163.105.0/24 dev eno1 proto kernel scope link src 10.163.105.238 metric 101
192.168.1.0/24 dev enp5s0 proto kernel scope link src 192.168.1.100 metric 105
192.168.1.0/24 dev enp1s0f1 proto kernel scope link src 192.168.1.21 metric 106
192.168.122.0/24 dev virbr0 proto kernel scope link src 192.168.122.1

      A higher metric means worse priority.

      Then, it complained that it couldn't find T1:

      [root@dhcp-10-163-105-238 amc13StandaloneMAN_2014-05-12]# bin/AMC13Tool -u
Connecting to AMC13...
T2 ip 192.168.1.188
T2 ip 192.168.1.188
T1 ip 192.168.1.189
T1 ip 192.168.1.189
Serial Number 33
T2 URI: ipbusudp-2.0://192.168.1.188:50001
T2 Address Table: file://map/AMC13_AddressTable_S6.xml
T1 URI: ipbusudp-1.3://192.168.1.189:50001
T1 Address Table: file://map/AMC13_AddressTable_K7.xml

***WARNING! T1 NOT FOUND AT THIS LOCATION!***
      [root@dhcp-10-163-105-238 amc13StandaloneMAN_2014-05-12]# bin/AMC13Tool -u
Connecting to AMC13...
T2 ip 192.168.1.188
T2 ip 192.168.1.188
T1 ip 192.168.1.189
T1 ip 192.168.1.189
Serial Number 33
T2 URI: ipbusudp-2.0://192.168.1.188:50001
T2 Address Table: file://map/AMC13_AddressTable_S6.xml
T1 URI: ipbusudp-1.3://192.168.1.189:50001
T1 Address Table: file://map/AMC13_AddressTable_K7.xml

***WARNING! T1 NOT FOUND AT THIS LOCATION!***

      To fix that, I had to just "re-apply" the config that set the ports

      cd /home/installation_testing/packages/experiment/lxedaq/amc13/amc13_v1_2_18/dev_tools/amc13Config/
./applyConfig -n 33
      cd /home/installation_testing/packages/experiment/lxedaq/amc13/amc13_v1_2_18/dev_tools/amc13Config/
./applyConfig -n 33

    2. Run amc13_scripts/ky_init.amc from inside AMC13Tool

      [root@dhcp-10-163-105-238 amc13StandaloneMAN_2014-05-12]# bin/AMC13Tool -u
Connecting to AMC13...
T2 ip 192.168.1.188
T2 ip 192.168.1.188
T1 ip 192.168.1.189
Serial Number 33
T2 URI: ipbusudp-2.0://192.168.1.188:50001
T2 Address Table: file://map/AMC13_AddressTable_S6.xml
T1 URI: ipbusudp-2.0://192.168.1.189:50001
T1 Address Table: file://map/AMC13_AddressTable_K7.xml

Pick an action (h for menu): fv
Virtex Firmware Version: 0x8127
Spartan Firmware Version: 0x2d

Pick an action (h for menu): do amc13_scripts/ky_init.amc
Entering script file amc13_scripts/ky_init.amc
*** Both chips have been issued a reset ***
Enabling AMC inputs from list: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11
Link status: 04100fff
Enable Fake Event Generator
'CONTROL1': 81270081

*****AMC13 Status*****
Status display detail level: 1
Control 0: 4b000048
  Monitor Buffer Empty
  TTC BCNT Error
Control 1: 81270081
  Create fake event at L1A
  Run Mode
AMC Link Status: 0fff0fff
  AMC13 Enabled Inputs: 00, 01, 02, 03, 04, 05, 06, 07, 08, 09, 10, 11
  AMC Input links locked: 00, 01, 02, 03, 04, 05, 06, 07, 08, 09, 10, 11
AMC Port Status: 0fff0fff
  AMC Link Versions incorrect: 00, 01, 02, 03, 04, 05, 06, 07, 08, 09, 10, 11
  Unsynced AMC Ports: 00, 01, 02, 03, 04, 05, 06, 07, 08, 09, 10, 11
AMC Bc0 Status: 00000000
  --No BC0s locked--
EVB Counters:  (All 32-bit counters read 0x0)
                   TTC BC0 err [0044]: 00000000 00000371
                      Run time [0048]: 00000000 00314f8f
                    Ready time [004a]: 00000000 00222012
                     Busy time [004c]: 00000000 000fff01
            L1A ovfl warn time [0050]: 00000000 00000001
AMC Counters:
                                       <---Link 00-----> <---Link 01-----> <---Link 02-----> <---Link 03-----> <---Link 04-----> <---Link 05----->
                                       *******************************************All counters read 0x0*******************************************


                                       <---Link 06-----> <---Link 07-----> <---Link 08-----> <---Link 09-----> <---Link 10-----> <---Link 11----->
                                       *******************************************All counters read 0x0*******************************************
Writing to T1:
  00000001: 00000000
Writing to T1:
  00000003: 00001fff
Writing to T1:
  00000001: 00000187
Writing to T1:
  0000001c: 80008000
Writing to T1:
  00000018: 0003fff0
Writing to T1:
  00000000: 00000010

Pick an action (h for menu): ^C
      [root@dhcp-10-163-105-238 amc13StandaloneMAN_2014-05-12]# bin/AMC13Tool -u
Connecting to AMC13...
T2 ip 192.168.1.188
T2 ip 192.168.1.188
T1 ip 192.168.1.189
Serial Number 33
T2 URI: ipbusudp-2.0://192.168.1.188:50001
T2 Address Table: file://map/AMC13_AddressTable_S6.xml
T1 URI: ipbusudp-2.0://192.168.1.189:50001
T1 Address Table: file://map/AMC13_AddressTable_K7.xml

Pick an action (h for menu): fv
Virtex Firmware Version: 0x8127
Spartan Firmware Version: 0x2d

Pick an action (h for menu): do amc13_scripts/ky_init.amc
Entering script file amc13_scripts/ky_init.amc
*** Both chips have been issued a reset ***
Enabling AMC inputs from list: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11
Link status: 04100fff
Enable Fake Event Generator
'CONTROL1': 81270081

*****AMC13 Status*****
Status display detail level: 1
Control 0: 4b000048
  Monitor Buffer Empty
  TTC BCNT Error
Control 1: 81270081
  Create fake event at L1A
  Run Mode
AMC Link Status: 0fff0fff
  AMC13 Enabled Inputs: 00, 01, 02, 03, 04, 05, 06, 07, 08, 09, 10, 11
  AMC Input links locked: 00, 01, 02, 03, 04, 05, 06, 07, 08, 09, 10, 11
AMC Port Status: 0fff0fff
  AMC Link Versions incorrect: 00, 01, 02, 03, 04, 05, 06, 07, 08, 09, 10, 11
  Unsynced AMC Ports: 00, 01, 02, 03, 04, 05, 06, 07, 08, 09, 10, 11
AMC Bc0 Status: 00000000
  --No BC0s locked--
EVB Counters:  (All 32-bit counters read 0x0)
                   TTC BC0 err [0044]: 00000000 00000371
                      Run time [0048]: 00000000 00314f8f
                    Ready time [004a]: 00000000 00222012
                     Busy time [004c]: 00000000 000fff01
            L1A ovfl warn time [0050]: 00000000 00000001
AMC Counters:
                                       <---Link 00-----> <---Link 01-----> <---Link 02-----> <---Link 03-----> <---Link 04-----> <---Link 05----->
                                       *******************************************All counters read 0x0*******************************************


                                       <---Link 06-----> <---Link 07-----> <---Link 08-----> <---Link 09-----> <---Link 10-----> <---Link 11----->
                                       *******************************************All counters read 0x0*******************************************
Writing to T1:
  00000001: 00000000
Writing to T1:
  00000003: 00001fff
Writing to T1:
  00000001: 00000187
Writing to T1:
  0000001c: 80008000
Writing to T1:
  00000018: 0003fff0
Writing to T1:
  00000000: 00000010

Pick an action (h for menu): ^C

Test Connection:

  1. Turn off enp5s0 as it will steal traffic from the 10GbE port on the same network
    ifdown enp5s0
    ifdown enp5s0
  2. Make sure enp1s0f1 is up, try pinging
    ifup enp1s0f1
[root@dhcp-10-163-105-238 amc13StandaloneMAN_2014-05-12]# ping 192.168.1.32
PING 192.168.1.32 (192.168.1.32) 56(84) bytes of data.
64 bytes from 192.168.1.32: icmp_seq=1 ttl=64 time=0.161 ms
64 bytes from 192.168.1.32: icmp_seq=2 ttl=64 time=0.093 ms
64 bytes from 192.168.1.32: icmp_seq=3 ttl=64 time=0.058 ms
    ifup enp1s0f1
[root@dhcp-10-163-105-238 amc13StandaloneMAN_2014-05-12]# ping 192.168.1.32
PING 192.168.1.32 (192.168.1.32) 56(84) bytes of data.
64 bytes from 192.168.1.32: icmp_seq=1 ttl=64 time=0.161 ms
64 bytes from 192.168.1.32: icmp_seq=2 ttl=64 time=0.093 ms
64 bytes from 192.168.1.32: icmp_seq=3 ttl=64 time=0.058 ms